Skip to main content

Full text of "Implementing a care level and timeliness review program"

See other formats



March 1978 









1978 ' 





Planning and Conditional PSROs and Support 
Center Organizations; Statexvlde Councils; 
Regional PSRO Project Officers 

DATE: MAR 2 1 1973 ■ 

Technical Assistance 
Document No. 14 



SUBJECT: Implementing a Care Level and Timeliness Review Program 


This technical assistance docviment describes a methodology — Care 
Level and Timeliness Review (CLTR) developed by the consulting firm 
McKlnsey and Company as a result of their previous work with the ''- 
Veterans' Administration. Over the last year and a half, CLTR has 
been tested under HSQB contract number 240-76-0071 in three hospitals 
under PSRO review in the Delmarva and Prince George's Foundations 
for Medical Care, Inc. An overview of the CLTR process and the 
experience of the test sites was presented to the National 
Professional Standards Review Council at their March meeting. The 
Council encouraged its further implementation In other PSROs and 
asked for feedback. Accordingly, we encourage PSROs to apply CLTR 
and provide us with comments regarding its usefulness in their area. 

CLTR follows the generic approach of the Medical Care Evaluation (MCE) 
study In order to assess admission and continued stay appropriateness 
and the timeliness of the care provided to hospitalized patients. 
It identifies that portion of total Inpatient days during the review 
period that could have been avoided if the hospital's admission, 
discharge, and treatment processes had worked as expected. The 
analysis Is based on a retrospective review of a 200 patient record 
sample chosen randomly from all hospital discharges during the review 
period. Each record is reviewed against two types of criteria: 
a) PSRO concurrent review screening criteria to determine whether 
days of stay are avoidable due to inappropriate admissions and 
continued stays and b) "turnaround time" criteria to determine whether 
the timeliness with which care is provided impacts on length of stay. 
In setting the latter criteria, turnaround times are developed for 
those diagnostic/therapeutic actions and hospital services which, if 
not performed in a timely fashion, could Influence length of stay. 
The results of this analysis are extrapolated to the hospital as a 
whole to identify the extent and apparent causes of avoidable days 
of stay within the hospital. These results likely will suggest 
problem areas requiring further study in order to pinpoint the 
underlying causes and to develop feasible Improvement actions. 

Planning and Conditional PSROs and Support Page 2 

Center Organisations; Statewide Councils; - ■ ^ - 

Regional PSRO Project Officers ' ' " ' ' ' ■ ■ ' • 

CLTR should be viewed as an analytical tool which builds upon and 
can be used in conjunction with other PSRO information and activities 
in order to meet program objectives. The initial results of CLTR 
can indicate problem areas where further analysis, e.g., profile 
analysis, indepth medical records review, may be warranted to 
identify where review should be focused. This analysis can lead to 
establishing measurable objectives and targeting PSRO monitoring to 
determine progress in resolving problems specific to the individual 
hospital. Similarly, CLTR can be part of the process whereby PSROs 
assess and document their impact at the local level. 

CLTR may be carried out as an MCE study and can count towards the 
minimum number requirements set by HSQB, or it can be performed as 
part of a PSRO's responsibility for assessing and monitoring delegated 
and nondelegated review. When carried out as an individual hospital 
MCE study, costs would be assigned to the delegated or nondelegated 
hospital MCE study function. When performed as an areavrLde MCE study or 
as part of the PSRO's assessment and monitoring of delegated and 
nondelegated review, costs would be assigned to areawide hospital 
review costs. 

The attached document provides detailed guidance and instructions on 
how to perform CLTR. In addition, 1-day workshops will be organized 
by the American Association of Professional Standards Review 
Organizations (AAPSRO) , as part of its expert assistance contract ■ 
with HSQB, to provide training to interested PSROs. Further 
communication on the dates and locations of these workshops will be 
forthcoming through AAPSRO. ... . 

y'-' ' Michael J. Goran, M.D. 

Attachment . _ S , . 




Prepared by McKinsey &c Company, Inc. 
• Pursuant to Contract # HSA-240-76-0071 

* March 1978 





Designate Key Participants „...,..... . ^. -.^ .i-: ^ 

Process Leader 

Records Analyst . - .v. • . 

Physician Advisor ® 


Administrative Liaison . ,.. . , , - a • ° 

Set Review Criteria .. . , ' 

Level of Care ^ 

Timeliness of Care . . : , . . . , ^ 

Other Criteria 1^ 

Categorize Possible Causes of Avoidable Days 12 

Specify the Sample Structure 13 

Select the Sample Cases .. . , . - . ,.. . 15 

Conduct Records Review 1^ 

Evaluating Necessity for Admission .. 16 

Reviewing Timeliness of Care 18 

Summarize Results and Extrapolate to Hospital ,, 22 

Discuss Findings with Hospital Leadership 23 


Identify Underlying Causes 24 

Physician Management Delays 2 5 

Ancillary Service Delays ' ■ • ' .. ^ . ^ .• 26 

Nursing Home Placement 26 

Outpatient Substitution • .■ :• . - . i 27 

More Frequently Available Services 28 

Establish Improvement Objectives 28 

Develop and Gain Agreement on Specific Corrective Actions 31 



1 - Care Level and Timeliness Review ' ' 33 

Illustrative Timetable and Staff Hours 

2 - Target Turnaround Times 34 

3 - Summary of Causes of Potentially Avoidable - 3 5 

Hospital Days 

4 - Admissions /Discharges List for Sample Selection 36 

5 - Record Review Worksheet • : ' • 37 

6 - Medical Records Analysis Summary ' ■ 38 

7 - Record Review Worksheet, Example 1 ' • 39 

8 - Record Review Worksheet, Example Z ... . - 40 

9 - Results of Records Analysis, Hospital A 41 

10 - Results of Records Analysis, Hospital B 42 

11 - Results of Records Analysis, Hospital C •■ . 43 

12 - Routine Inpatient X-Ray Process 44 

13 - Hospital Corrective Action Plan 45 




Exhibits to the Appendix 

A - 1 - Size of Confidence Interval vs. Sample Size 50 
for Various Avoidable Days Percentages 

A - 2 - Size of Confidence Interval over Range of 51 
Avoidable Days Percentages for a 200-Case 

A - 3 - Random Number Table 52 





The purpose of this document is to provide PSROs and hospitals 
the information needed to carry out a supplementary approach to 
utilization review - Care Level and Timeliness Review (CLTR), 
which has recently been tested in two PSROs and three hospitals 
under the auspices of the Health Standards and Quality Bureau 
(HSQB). CLTR is fundamentally a form of medical audit dealing 
with the appropriateness of admissions and continued stays and 
the timeliness of the care provided to those people who are hos- 
pitalized. Its promise lies in its potential for helping PSRO and 
hospital managements (1) determine the extent and underlying 
causes of unnecessary hospital utilization, (2) develop goals and 
corrective actions for reducing avoidable hospital days, and 
(3) evaluate progress in implementing these actions. 

The concept of CLTR is based on periodic, retrospective review 
of a statistically representative sample of patient records. For 
each patient whose record is included in the sample, appropriate- 
ness of admission and readiness for discharge can be reviewed 
against the same criteria used for admission certification and 
continued stay review in current PSRO programs. For those pa- 
tients whose admissions are appropriate, the timeliness of the 
care process applied to them during hospital stay is reviewed - 
but not against diagnosis -specific length of stay norms such as 
those used in the continued stay review program. Instead, tar- 
get "turnaround times" for frequently occurring care events are 
defined by the PSRO and hospital staffs. Then, for each record 
the reviewer: (1) identifies any care delays in relation to these 
turnaround times, (2) evaluates the impact, if any, of such care 
delays on inpatient hospital days, and (3) determines the appar- 
ent causes of such delays. 

The aggregated results of the review of the records sample are 
then extrapolated to the hospital as a whole to provide an esti- 
mate of: (1) the portion of inpatient days during the review per- 
iod that potentially could have been avoided if the hospital's 
admission, discharge, and treatment processes had worked as 
well as expected; and (2) the areas in which action to improve 
care processes might be taken. These extrapolated results 

- 1 - 

likely will suggest problem areas in which more intensive studies 
should be undertaken. Once any further studies are completed, 
targeted action programs to avoid unnecessary patient days and 
improve patient care can be developed, costed out, and, when 
feasible, implemented. 

CLTR is not designed to evaluate the appropriateness or quality 
of services provided to patients once the medical necessity of 
admission has been determined. However, review of the records 
sample may be used to measure the incidence of generic care prob- 
lems not directly related to admission/discharge appropriateness 
or care timeliness - i. e. , conditions that are not diagnosis- 
specific but cut across the case mix of the hospital. Drug reac- 
tions might be an example of such problems. Similarly the records 
review could test for adherence to certain hospital policies, such 
as timely entry of discharge summaries. -i -,, , ' ■< • 

The remainder of this chapter describes a number of important 
roles CLTR can play in helping PRSOs and hospitals improve the 
utilization of inpatient hospital services. These roles include: 
(1) developing programs to improve care timeliness; (2) focusing 
review procedures to concentrate on problem areas; (3) screen- 
ing hospitals for exemption from concurrent review procedures; 
(4) setting objectives for and measuring utilization improvements; 
and (5) supporting health care planning efforts. 

- 1. Improving care timeliness . Although it incorporates 
review of admission and discharge practices, CLTR's unique 
contribution is the capability it provides for evaluating the 
, timeliness of care rendered in a hospital - in terms of both 
the expedition with which individual hospital functions affect- 
ing patients' lengths of stay are carried out and the time- 
liness with which physicians take action on the information 
available to them in treating their patients. The process is 
specifically designed to both identify systematic delays in a 
hospital's care processes and to point to the causes for those 
delays. With such information in hand, hospital and PSRO 
managers can develop and carry out action programs aimed 
. . at correcting the problems leading to delays in the care 
process. Earlier experience with CLTR, as well as the 
; recent test in three diverse cormnunity hospitals associated 
;. with two PSROs, has demonstrated its effectiveness for 
this purpose. . ^ . . . • w;, , = ■ : 

- 2 - 

2. Focusing review procedures on important problems. 
CLTR records sample review and follow-up studies can 
provide reliable information for focusing concurrent review 
procedures, profile analyses, and other care program im- 
provement studies on high-potential problem areas: 

y For example, CLTR might show that inappropriate admis- 
sions are not a significant problem for a hospital, but that 
failure to discharge when maximum hospital care benefit had 
been achieved is contributing importantly to avoidable days of 
hospitalization. Under these circumstances, the PSRO might 
concentrate on continued stay review, even instituting more 
frequent reviews, while scaling back or eliminating admis- 
sions certification. 

IF Alternatively, where CLTR suggests that insufficient sub- 
stitution of outpatient for inpatient care and too little pre- 
admission work-up are problems, steps could be taken in the 
concurrent review process to strengthen admission review - 
e.g. , tightening admission criteria or establishing preadmis- 
sion review for certain cases categorized by diagnosis, in- 
tended surgical procedure, or physician. 

y A conclusion from CLTR that "physician management" 
problems are contributing importantly to avoidable days of 
hospitalization could trigger a profile analysis aimed at 
validating this finding and pinpointing physicians or services 
for which the problem may be particularly acute. Indeed, it 
may be possible from the data developed in the CLTR to 
identify portions of the physician staff or hospital case mix 
where these problems appear most troublesome to provide a 
basis for focusing the profile analysis. In either case, the 
result might be a decision to concentrate continued stay re- 
view on those physicians or case types offering the greatest 
opportxinity for reduced hospital utilization. 

f A records sample review and follow-up studies might in- 
dicate that physician delay in ordering discharge is a system- 
atic problem for a group of physicians or type of diagnosis. 
In such cases more intensive continued stay review along with 
peer counseling might be an appropriate corrective action. 

3. Screening for exemption from concurrent review. Just 
as CLTR can be used to identify areas where PSRO effort 
should be concentrated, the results can also be used along 
with other PSRO information to help reach decisions to exempt 
hospitals from concurrent review. Hospitals that evidence 
few avoidable patient days, or for which well-conceived action 
programs are developed to correct the causes of the avoid- 
able patient days identified, could be relieved of concurrent 
review requirements. In the latter case, PSRO and hospital 
management could concentrate until the next CLTR review on 
ensuring that action programs developed on the basis of the 
preceding records sample study were being pursued and the 
results documented. In either case, CLTR could be conduct- 
ed on an annual basis under direction or with the monitoring 
of the PSRO in order to assure the continued appropriateness 
of the decision. If exemptions thresholds are appropriately 
set, overall program cost-effectiveness can be better served 
by exemption of a hospital under specified circumstances 
than by continued application of relatively costly concurrent 
review procedures. , ..... . . .• . , 

4. Setting objectives for and measuring utilization improve- 
ment. The results of a CLTR study at a hospital can be 
agreed upon as a target for the coming year - expressed in 
terms of a reduction in avoidable days to be achieved. Then, 
as long as fairly comparable records sample review proce- 
dures (sampling technique, turnaround times, and the like) 
are employed, and the mission of the hospital does not change 
substantially in the interim, the improvement potential of a 
hospital identified in a subsequent review can fairly be com- 
pared with these objectives to determine whether progress is 
being made in eliminating avoidable days of hospitalization. 
Between CLTR studies, the PSRO can monitor progress in 
carrying out agreed upon improvement actions. Thus, the 
CLTR can provide a basis for establishing objectives for 
utilization improvement, and for measuring progress in 
accomplishing those objectives at the level of the individual 
hospital or across all hospitals within a PSRO area. 

5. Supporting health care planning activities. CLTR provides 
aggregate information on the efficiency with which health 
care resources are utilized, and thus can be used to identify 

- 4 - 

potential savings in inpatient days and associated operating 
costs and beds. Such information can be valuable to health 
care planners and other health related agencies. Health 
Systems Agencies and state planning agencies could use in- 
formation from periodic CLTR studies as one basis for reach- 
ing decisions on facilities expansions or retrenchments, and 
for guiding service capacity and equipment actions by hospi- 
tals and by long-term care facilities, free-standing clinics, 
and other providers to promote appropriate hospital utiliza- 
tion. Similarly, as rate-setting and capitation reimbursement 
programs develop across the country, data from CLTR studies 
could provide useful information to agencies involved in these 
activities. Use of such findings by other agencies - for ex- 
ample, for budget- setting - should help provide incentives for 
hospital leadership to devise and implement action programs 
to reduce the level of avoidable hospital days identified by 
CLTR. . , 

* * * 

The remainder of this document presents the steps to be followed 
in implementing CLTR. 

- 5 - 


CLTR has two phases. This chapter deals with the first phase, 
analysis of a hospital's care activities through retrospective 
review of a representative sample of recent medical records. 
During this phase, the extent of the potentially avoidable hospital- 
ization incurred at a hospital is determined in total and by major 
apparent cause. The next chapter describes the second phase - 
the development of realistically achievable action plans designed 
to correct the most significant underlying causes of unnecessary 
hospitalization. _ . ^ , 

The care analysis phase of CLTR has eight steps: (1) designate 
the key participants, (2) set review criteria, (3) categorize pos- 
sible causes of avoidable days, (4) specify the sample structure, 
(5) select the sample cases, (6) conduct the records review, (7) 
summarize the results and extrapolate to the hospital, and (8) 
present findings to hospital leadership. 


There are four roles to be played in carrying out CLTR: (1) pro- 
cess leader, (2) records analyst, (3) physician advisor, and (4) 
administrative liaison. The same individual may play more than 
one role; no role requires full-time effort over an extended period 
of time. Together, the process leader, physician advisor, and 
administrative liaison form what could be termed the CLTR 
management group. 

Exhibit 1 illustrates the timetable and staff time requirements for 
a typical application of CLTR. In the example, fewer than 500 
hours of PSRO and hospital staff time are required over a 20-week 
period. Of course, the time requirements and scheduling may 
vary considerably from hospital to hospital, depending on such 
factors as (1) the clarity of the medical records; (2) the extent 
and complexity of identified causes of avoidable days; (3) the ease 
with which appropriateness of admission and discharge criteria 
are applied; and (4) the intrusion of other PSRO or hospital priorities 
on staff time. The following paragraphs briefly describe each of 
the key roles in the CLTR process. 

- 6 - 

Process Leader. The CLTR process leader, who should be a 
member of the PSRO staff, has three principal responsibilities, 

1. Directing and monitoring the application of CLTR in each 

2. Carrying out some of the more complex and important 

3. Presenting interim and final findings to PSRO, hospital, 
and other audiences, and assisting in developing corrective 
action programs. 

Thus, the process leader is ultimately responsible for the effec- 
tiveness of CLTR in significantly reducing medically unjustified 
hospitalization. He or she should be prepared to spend about 
1 day per week over a 15- to 25-week span in applying CLTR in a 
single hospital. 

Records Analyst, The records analyst's chief responsibility is 
the review of all sample charts, in conjunction with the physician 
advisor where necessary, to determine avoidable hospital days, 
A single chart should take 20 to 60 minutes to review, depending 
on its complexity, documentation, and extent of apparently unnec- 
essary hospitalization. The records analyst also carries out other 
clerical or analytical tasks related to the records review, such as 
arranging for the records to be drawn, summarizing results, and 
carrying out follow-up analyses of the records or of other data. 

The records analyst, who could be a member of either the PSRO 
or hospital staffs, should be familiar with medical records and 
terminology, and should be able to be judicious and thorough in 
reviewing a large sample of records. In the recent demonstration 
of CLTR in three hospitals, the records analysts were an experi- 
enced records review coordinator, a medical care evaluation (MCE) 
analyst with a background in pharmacology, and a third-year 
medical school student; all were paid by a PSRO. In another setting, 
medical residents were effective records analysts. The records 
review can be scheduled flexibly to accommodate the records 
analyst's other responsibilities. 

Physician Advisor. The physician advisor is charged with main- 
taining the medical integrity of the CLTR records review process. 
His or her participation is critical in establishing review criteria, 
evaluating physician decision-making in complex or inadequately 
documented cases, developing feasible plans to correct apparent 
problems in physicians' case management and in the overall func- 
tioning of the hospital, and gaining understanding of and support 
for action plans among hospital staff. 

The physician advisor should expect to review 10 to 20 percent of 
the 200 cases sampled, both to validate the records analyst's 
findings with respect to straightforward cases and to provide 
medical expertise in more complex ones. Each case reviewed 
should take 10 to 20 minutes, since the records analyst will have 
diagrammed the significant case events ahead of time. Altogether, 
the physician advisor should expect to devote 1 to 2 hours a week 
to the CLTR process in one hospital. 

Administrative Liaison. The administrative liaison represents 
hospital management on the CLTR management group. He/she 
also should provide administrative support to the process (e.g. , 
arranging for work space and medical records department assis- 
tance), advise on care timeliness criteria in administrative areas, 
help to analyze underlying causes of apparent delay, and develop 
with the process leader and physician advisor cost-effective, 
implementable corrective action plans. An assistant director 
played this role in two of the three hospitals in which CLTR was 
recently tested. The records analyst performed this function in 
the third, and smallest, hospital. 

The administrative liaison should have the support of hospital 
leadership and should understand hospital operations. He/she 
should be able to locate necessary hospital financial and opera- 
tional data, facilitate examination of functions suspected of 
causing care delays, and be able to work with hospital leadership 
to implement proposed corrective action plans. 

The time required of the administrative liaison is minimal until 
CLTR's second phase - pinpointing causes of avoidable days and 
developing corrective action plans. One or two days a week over 
1 to 3 months may be required during this phase. 

- 8 - 

2. SET 


At least two types of review criteria must be agreed on: (1) admis- 
sion and discharge - "level of care" - appropriateness; and (2) 
timeliness. If CLTR is to be used to evaluate not only care level 
and timeliness but also the appropriateness of certain inpatient 
care practices, adherence to certain administrative policies, or 
frequency of specific care problem indicators, review criteria 
are needed in these areas as well. This section describes these 
criteria more fully and suggests how they should be developed. 

Level of Care. Level of care criteria assist the records analyst 
in determining: (1) whether a patient should have been hospitalized 
in the first place; (2) if so, whether there were tests or proce- 
dures that should have been carried out before admission; and 
(3) when the patient should have been discharged. Many PSROs 
will choose to employ the identical criteria used in concurrent 
review in assessing appropriateness of admission and discharge - 
for example, the American Medical Association's Sample Criteria 
for Short-Stay Review as altered to reflect local practice patterns. 

Timeliness of Care. Criteria for timeliness of care, or 'target 
turnaround times," are the elapsed times between the initiation 
of the order for a diagnostic or therapeutic procedure or test and 
the performance of that procedure or the availability of test results. 
Turnaround times used in the recent application of CLTR in three 
hospitals are set out in Exhibit 2. There are two main steps in 
developing target turnaround times: 

1. Agree upon the diagnostic and therapeutic actions for which 
timeliness of care criteria are to be established. The CLTR 
process leader and/or administrative liaison should meet 
individually or in a group with the heads of those departments 
responsible for key diagnostic and therapeutic services 
(radiology, clinical laboratories, social work, surgical sched- 
uling, and administration) to agree on the diagnostic or 
therapeutic actions or procedures which, if not performed in 
timely fashion, could influence length of stay. As illustrated 
by the timeliness criteria developed for the three hospitals in 

- 9 - 

the recent CLTR demonstration (see Exhibit 2), the actions 
may include (a) physician's diagnostic or therapeutic inter- 
vention in the patient's course of care, (b) laboratory tests, 
diagnostic X-ray procedures, EKGs, and EEGs, (c) consult 
responses, (d) surgical procedures, (e) discharge planning 
activities, including nursing home placement and arrange- 
ment of suitable home environments, and (f) administrative 
discharge processing. Events included in the list should have 
two qualities: first, they should be capable of causing an 
impact on length of stay; secondly, they should be traceable 
from the medical charts. 

2. Establish the target turnaround times. The process leader, 
administrative liaison, and physician advisor should work with 
other hospital staff members as appropriate to establish target 
turnaroiind times. These performance criteria should reflect: 

a. Normal practice regarding "availability to the physician. " 
That is, turnaround time for routine X-ray procedures should 
be measured from the time the doctor's order is written to 
the time results are available to that doctor. Nevertheless, 
the "time results are available" can mean different things, 
depending on when and how hospital medical staff commonly 
review results from radiology: (1) when the film is developed 
and available for reviewing in the X-ray department; (2) when 
films are verbally reviewed during X-ray rounds at day's 
end; (3) when the film has been evaluated by a radiologist or 
technician and a report dictated, transcribed, and delivered 
to the ward; or (4) when the transcribed report has been filed 
in the patient's chart. The CLTR management group and 
department heads must decide which of these times is most 
pertinent and whether the measure chosen can be determined 
from reviewing the sample charts. 

b. The hospital's accepted standards for procedures . Turn- 
around times should reflect the elapsed times for procedures 
normally expected within the hospital for each service when 
it is functioning smoothly. They should not represent the 
longest or shortest expected times, nor need they represent 
statistically determined mean turnaround times. Rather, 
they should be based on reasonable expectations according to 

- 10 - 

the best judgment of the CLTR management group and depart- 
ment heads. For example. Exhibit 2 shows, for Hospital A 
during the recent CLTR demonstration, that routine laboratory 
tests ordered in the morning were expected back the same day. 
When the test was ordered in the afternoon, results were 
expected back the next weekday. Therefore, that hospital 
set its target turnaround time for this kind of procedure as 
"same or next weekday. " 

c. More ambitious but realistically achievable turnaround 
times than currently accepted standards for activities whose 
pace might be improved. Turnaround times set according to 
accepted standards when compared to actual times on the 
medical charts reflect the extent to which the hospital meets 
its own performance expectations. But there may also be 
certain departments or procedures which the CLTR manage- 
ment group or department heads believe realistically could 
be more responsive, e.g. , where a hospital's accepted turn- 
around times for one department are much longer than those 
for other departments in the same institution or than those of 
similar departments in other institutions. Improvements in 
such departments' responsiveness might hasten a patient's 
care without compromising its quality and, in some instances, 
even enhancing it. If such a procedure is identified, a second, 
more desirable turnaround time should be devised and applied 
along with the accepted standard. 

For instance, a hospital may wish to determine the extent to 
which limited special X-ray scheduling prolongs its patients' 
lengths of stay. To illustrate: if certain special X-rays 
were performed only Tuesday and Thursday, a target turn- 
around time set as "the next available Tuesday or Thursday 
after the exam is clinically indicated" would serve only to 
show how efficiently the hospital operates based on its cur- 
rent scheduling capabilities. A turnaround time for special 
X-rays set as "same or next weekday" would allow reviewers 
to measure the extent to which current scheduling capabilities 
lead to avoidable hospitalization days. Where desired turn- 
around times differ from turnaround times that are reasonable 
under current hospital practices, two target turnaround times - 

- 11 - 

"desired" and "normal under current practices" - should be 
used in evaluating possible avoidable days. An explanation 
of how both sets of criteria can be used is provided in a 
later section, "Reviewing Timeliness of Care." 

Other Criteria. As mentioned in Chapter I, the sample of cases 
to be reviewed retrospectively for care level and timeliness can 
also be reviewed for other problems that cut across a hospital's 
case mix. For example, PSRO or hospital leadership may wish 
to monitor other generic indicators of care problems such as 
infection incidence, drug reactions, patient accidents - in short, 
any indexes of the effectiveness or appropriateness of the hospital's 
care activities that most effectively can be explored through a 
hospital-wide sample of patient records designed to provide 
statistically meaningful information upon which to base corrective 

Similarly, PSRO and hospital management may wish to use the 
record review procedure to assess adherence to certain adminis- 
trative procedures. These might include whether: (1) admission 
notes are written within 24 hours of admission; (2) physical 
examination is performed and noted on the chart within 48 hours 
of admission; (3) patient history is taken within 48 hours; (4) pro- 
gress notes are added to the chart at least every 3 days; and 
(5) the discharge summary is written within 15 days of discharge. 

In either case, the necessary step is for the CLTR management 
group and the hospital staff to agree upon the criteria to be applied 
to the record sample. , 



Another step necessary in preparing for case review is establish- 
ing categories for causes of avoidable days. These categories 
should be as specific as possible so that follow-up analyses after 
the records review, typically needed to provide a firm basis for 
developing corrective action programs, can be as limited and 
focused as possible. However, the categories should not be so 
specific that evidence in the medical records will be inadequate 
to show how a delay should be classified. All avoidable days 

- 12 - 

should fall into only one category. The categories and sub- 
categories employed in the recent demonstration are provided 
in Exhibit 3. 

In some applications of CLTR, cause-of-delay categories might 
be defined to accommodate examination of the potential impact on 
avoidable inpatient days of alternative hospital operating policies 
or procedures. For example, in one hospital with access to a 
urologist on only Tuesdays and Thursdays, it might be appropriate 
to establish a specific delay category to reflect this limited capa- 
bility. In this way extra days of stay which arose because the 
urologist was not available could be differentiated from those 
arising because he failed to respond when he was available. Pre- 
sumably, the presence of many avoidable days due to the first 
cause would justify scheduling additional urology coverage, while 
many days in the second cause would suggest the need to work 
with the urologist to determine whether perhaps (1) the consult 
request system is faulty, (2) he has too little time to respond to 
the consult requests he receives, or (3) he needs to be more 
diligent in responding to consult requests. 

If the records review is being used to evaluate the incidence of 
other problems in addition to avoidable days of hospitalization, 
such as generic care problems or adherence to certain hospital 
policies, categories for recording these findings will also be 


The sample of medical records should provide a representative 
cross- section of cases to enable reviewers to draw meaningful 
conclusions about overall hospital utilization and operations. 
For this reason, the sample structure should follow strict giiide- 
lines: (a) certain kinds of admissions and discharges should be 
omitted from the records sample because the care practices and 
effectiveness of the hospital may not consistently affect the hos- 
pital days incurred by patients in those categories; (b) certain 
time periods should be covered; and (c) a certain number of cases 
should be included in the sample to be reviewed. The following 
paragraphs describe steps required to achieve this structure: 

- 13 - 

a. Identify categories of cases with the following character- 
istics so that they may be excluded from the sample: 

y Stays not controlled by physicians or hospital manage- 
ment, e.g. , stays ending in death or discharges against 
medical advice 

y Programmed lengths of stay where hospitalization is 
rarely extended or foreshortened from program guide- 
lines - e. g. , admissions to 1-day dialysis or 2-week 
detoxification units 

^ Stays for which neither level of care nor length of 
stay is an issue - e. g. , normal deliveries, possibly 
psychiatric admissions 

b. Ensure that the time period covered by the sample repre- 
sents current practices as accurately as possible. To avoid 
any seasonal effects, cases ideally should be chosen from a 
full 12-month period. However, the sample period should 
avoid, where possible, periods of major interruptions or 
alterations in normal hospital services (e.g. , an operating 
room closed for repairs for 2 months) which might distort 
case mix or unfairly reflect delays in certain services. 
Finally, the time period should be recent enough so that it 
accurately reflects current care processes and does not 
identify problems for which solutions have already been found. 

c. Specify a sample size which balances statistical power and 
resource economy. The statistical power of the method for 
estimating the percentages of days that are avoidable - either 
collectively or in relation to a specific cause - is largely 
unaffected by the size of the hospital. Rather, statistical 
power is primarily a function of the sample size and the size 
of the avoidable-day percentage being estimated. As described 
in the appendix to this document, for most hospitals a sample 
of 200 records will achieve a reasonable balance in this regard. 

- 14 - 


In selecting the sample of medical records to be reviewed, reason- 
ably straightforward but specific procedures should be followed 
to ensure that the conclusions reached in reviewing the records 
can be usefully applied to the operations of the hospital as a whole. 
These procedures are described in detail in the appendix to this 
document. In summary, the following steps should be taken: 

1. Obtain a comprehensive list of discharges during the 
sample period. Discharges, rather than admissions, should 
be used to ensure that the sample records represent completed 
stays. The list should provide the medical records number 
for each patient, or some other information - such as the 
patient's name - from which the medical record describing 
the case can be identified. 

2. Using a set of random numbers, select 300 records. The 
appendix describes approaches for using random numbers to 
select the records sample. Initially selecting 300 records 
from the total list of patient records from the sample period 
should assure an ultimate sample of at least 200 records after 
diagnoses and dispositions deemed to be inappropriate for 
CLTR have been excluded. (If a hospital's patient mix reflects 
an unusually high proportion of cases categorized for exclusion 
from the sample, more than 300 records might be selected 
initially. ) 

3. Screen agreed-upon diagnoses and dispositions for exclu- 
sion from the 300-record sample. If the comprehensive list 
of discharges during the sample period provides sufficient 
information, records dealing with conditions that have been 
agreed to be excluded from the CLTR can be screened out 
before the records are pulled. Alternatively, the records 
themselves can be inspected as they are pulled to identify 
those to be excluded. Even if the comprehensive list provides 
information for screening for exclusions, during the course 
of the review the records analyst should perform a final check 
for any records that should be excluded. 

- 15 - 

4. Finalize the records sample. After screening for exclu- 
sions, naore than 200 records may remain from the initial 
sample of 300. All may be reviewed, or the first 200 cases 
selected may be taken as the sample. 

Final disposition of the initial 300 records sample should be docu- 
mented using the form illustrated in Exhibit 4. In the illustration, 
all cases except those checked under "Normal" are excluded from 
the final 200- case sample. 


The review of medical records is divided into two major parts: 
(1) evaluating the necessity for admission and (2) reviewing the 
timeliness of care and discharge where admission has been judged 
necessary. If other indexes of care program effectiveness or 
adherence to administrative policies are to be evaluated, they 
should be noted as the records are reviewed against care timelines 

In carrying out the records review, the records analyst should 
diagram and review every case, referring cases to the physician 
advisor: (1) if the records analyst cannot determine admission 
necessity or appropriateness of discharge timing, or (2) if the 
case is so complex that the records analyst cannot determine the 
"critical path" of care or the impact of care delays on length of 
stay. Physician advisor approval need not be sought in clear-cut 
instances of avoidable hospital days since - unlike the concurrent 
review process - CLTR does not result in payment denials. 

Evaluating Necessity for Admission . The records analyst should 
evaluate each sample chart to establish whether the admission was 
necessary. Three steps are needed to evaluate an admission: 

1. Fill out the top part of the Record Review Worksheet 
(see Exhibit 5). This part of the form includes an admission/ 
discharge identification number to facilitate reference if 
questions arise about review conclusions. It also provides 
for recording other information such as admitting and final 

- 16 - 

diagnoses, chief complaints, age, discharge destination, 
length of stay, surgical procedures, and similar data which 
may be helpful in follow-up studies to determine the specific 
underlying causes of unnecessary hospitalization. 

2. Assess appropriateness of admission based on admission 
criteria. In this step, the records analyst should determine 
whether the patient's indications supported admission. For 
instance, if the admitting diagnosis is "pernicious anemia, " 
the agreed indications for admission might read: "(a) symp- 
tomatic anemia (e.g. , chest pain, respiratory distress, 
weakness and fatigue, alterations of consciousness) or (b) 
neurologic deficit. "* If one of these criteria is satisfied by 
indications recorded in the chart, the analyst should judge 
the admission necessary and proceed with review of the time- 
liness of care. If one of the criteria is not found, the chart 
should be referred to the physician advisor for judgment on 
the necessity of admission. 

3. Record results on the Medical Records Analysis Summary. 
If, after review by the physician advisor, an admission is 
judged inappropriate, the records analyst should write 
''Admission Not Necessary" on the Record Review Worksheet 
and record the patient's total stay as avoidable days under 

the Medical Records Analysis Summary column marked 
"Patient Could Have Been Diagnosed or Treated on an Out- 
patient Basis" (see Exhibit 6). 

* - American Medical Association, Sample Criteria for Short - 

Stay Hospital Review, p. 55. 

* - Very occasionally, a patient may be admitted inappropriately 

but incur a condition while in the hospital that justifies 
admission. Each "inappropriate admission" record should 
be checked for such circumstance, and only those hospital 
days before the condition became evident should be counted 
as avoidable due to inappropriate admission; the remainder 
of the stay should be reviewed for timeliness of care and 

- 17 - 

Reviewing Timeliness of Care. Timeliness review involves mea- 
suring the potential for reducing the length of stay for an appro- 
priately admitted patient by improving the pace of the hospital's 
care processes. For cases whose admission is judged appropriate, 
timeliness review comprises five steps: (1) laying out the course 
of hospitalization for each patient, (2) identifying delays, (3) eval- 
uating any impact of these delays on length of stay, (4) identifying 
causes of the delays, and (5) entering findings on the Medical 
Records Analysis Summary. 

1. Laying out the course of hospitalization on the Record 
Review Worksheet. The hospital record of each patient 

. should be reviewed in detail to determine exactly what happen- 
ed, and when, during the course of care. All diagnostic 
and therapeutic procedures and tests should be recorded in 
time sequence from admission to provide the basis for sub- 
sequent evaluation of the timeliness of the care program. 
Special note should be made of the points at which a given 
diagnosis was ruled out, specific therapy was begun and ended 
for a given diagnosis, or complications, if any became evident 
or disappeared. These events represent medical milestones 
in a patient's hospital stay and have implications for the 
physician's management of the patient and for the patient's 
length of stay. For cases with multiple diagnoses or "rule- 
outs, " it is helpful first to note the points at which a diagnosis 
was ruled out and then to record those diagnostic and therapeu- 
tic procedures and tests related to the decision. 

2. Identifying delays in the course of hospitalization. The 
records analyst accomplishes this step by comparing the 
turnaround time taken for each procedure and test recorded 
on the Record Review Worksheet with the target turnaround 
time set in the initial stages of CLTR. In addition, the 
records analyst should determine whether the criteria for 
timely discharge for this diagnosis or procedure were met 
any sooner than the day on which discharge was ordered. If 
there are no apparent delays in hospital care or in discharge, 
the records analyst should note "No Delays" in the "Apparent 
Delays" column on the Record Review Worksheet. The sample 
number, admission/discharge number, primary diagnosis, 
and actual length of stay should be recorded on the Medical 

- 18 - 

Records Analysis Summary and "0" placed under "Total 
Number of Avoidable Days. " Analysis of the chart is then 
complete if there are no delays. 

If there are delays, the records analyst should record the 
number of days for each delay in the "Apparent Delays" 
column. Where both an accepted standard and a more ambi- 
tious turnaround time have been assigned, both the number of 
days' delay according to the first turnaround time and the 
additional delay resulting from applying the more ambitious 
turnaroiand time should be noted. No entry should be made 
on the Medical Records Analysis Summary yet, because 
the delays have not yet been evaluated for their actual impact 
on length of stay. 

3. Evaluating delays for their impact on length of stay. Ther 
are delays in many patients' courses of treatment. However, 
many of these delays do not result in avoidable days of hospi- 
talization because the patient had to be in the hospital for othe 
procedures that were underway. The relevant delays for 
CLTR are those \^ich prolong length of stay; they should be 
distinguished from those that do not prolong stay in order to 
determine the hospital's potential for reducing avoidable days 
of hospitalization across its patient load. The essential 
question to answer in determining length of stay impact is 
whether the patient could have been discharged earlier if the 
delay had not occurred. In determining the answer, the 
following questions should be considered: 

y Were the results of delayed diagnostic procedures 
critical to the next step in diagnosis, or to initiation or 
continuation of the patient's active treatment? 

y If there were delays in completing specific procedures 
did the necessity for ongoing therapy or treatment to 
alleviate the patient's illness during this period override 
any impact the delays might otherwise have had on length 
of stay? 

5" Was the patient's health status such that complications 
slowed his otherwise timely diagnosis or treatment? 

- 19 - 

Special note should be made here of two points. First, at no point 
in timeliness review should the records analyst presume more 
information than was available to the attending physician at the 
time he acted. Second, where length of stay seems prolonged and 
the patient appears stabilized, the records analyst usually should 
give the attending physician's judgment for continued stay the 
benefit of the doubt, unless discharge guidelines or the physician 
advisor suggest otherwise. 

The following case examples, based on the recent test, illustrate 
how delays in the courses of patients' care can be evaluated for 
their contribution to avoidable days of hospitalization: 

Example 1. The case diagrammed in Exhibit 7 is that of a 
71 -year-old diabetic who has suffered a stroke, resulting in 
left- side paralysis. The course of therapy was fairly routine 
for this sort of case, but a number of delays occurred, some 
of which lengthened this patient's hospital stay unnecessarily. 
A physical therapy consult was ordered on the day of admission, 
but was not answered and therapy begun until the third day of 
care - a 1-day apparent delay. On the day of admission, the 
attending physician realized that this patient would need nurs- 
ing home care and so ordered the Social Service Department 
to initiate placement planning. However, Social Service did 
not initiate the placement process (by filling out Form 256R) 
until Day 3 - a 1-day delay. Having mailed Form 256R to the 
Health Department the next weekday for approval, a process 
which typically takes a full day. Social Service reported on 
Day 8 that the patient was awaiting an open bed. On Day 14, 
Social Serxdce was notified that a bed would be available one 
week later, and on Day 21 the patient finally was transferred 
to a nursing home. The physician advisor, upon reviewing 
this case, determined that the patient had received maximum 
hospital benefit by Day 4. 

To determine what impact these days had on length of stay, 
let us consider how the course of care should have proceeded 
had delays not occurred. Physical therapy would have begun 
one day sooner, continued through the stay, and still have been 
needed after discharge. Social Service would have initiated 

- 20 - 

placement on Day 2, mailed the 256R to the Health Department 
on Day 3, and, having given the Health Department Day 6 
(Monday) to process the 256R, been able on Day 7 to place 
the patient in a nursing home if a bed had been available - 14 
days sooner than it actually occurred. The physical therapy 
delay did not lengthen hospitalization unnecessarily as it was 
continued at the nursing home anyway. Social Service's 
delay in initiating placement planning may have contributed 

1 day of delay, assuming that the number of days required 
for a bed to become available (once it is sought) is constant. 
The lack of a nursing home bed accounted for the remaining 
13 unnecessary days, since the patient could have been dis- 
charged on Day 8 even if all other delays still had occurred. 

Example 2. In the case diagrammed in Exhibit 8, a 70-year- 
old was admitted with elevated blood sugar and hallucinations. 
Diabetes and cerebral arteriosclerosis were the focus of care. 
Over 16 days, blood sugar levels were eventually stabilized, 
until tolinase could be lowered to a maintainence dose on 
Day 20. The records analyst - with confirmation by the 
physician advisor - felt that discharge was indicated on Day 21, 

2 days earlier than discharge occurred. This delay added 
unnecessarily to length of stay. However, other delays - the 
2-day delay in Dr. C's response to a consultation request and 
the 3-day delay in ordering physical therapy - had no apparent 
impact on length of stay since they occurred while the patient's 
blood sugar levels were still uncontrolled and did not delay 
any care leading to discharge. 

4. Identifying and recording the apparent causes of avoidable 
patient days. Identification of the specific causes for avoid- 
able days of hospitalization is the starting point for developing 
programs to reduce inappropriate hospital utilization. Critical 
delays in the course of care should be categorized by broad 
cause - primarily either medical or administrative - and 
subcategorized as shown in Exhibit 3. Thus, in the case set 
out in Exhibit 7, 1 day of unnecessary hospitalization was 
attributed to "Nursing Home Placement Delay - Social Service" 
and 13 days to "Nursing Home Placement Delay - Lack of Bed 
or Funding" - 14 days in all. In the second example diagrammed 

- 21 - 

in Exhibit 8, the 2 days of unnecessary hospitalization were 
attributable to the physician's delay in ordering discharge. 

For each case, the avoidable days - by cause and in total - should 
be entered in the appropriate column of the Medical Records 
Analysis Summary (Exhibit 6). 


After all records are reviewed, the CLTR process leader should 
summarize the sample results to estimate the proportion of in- 
patient days that are avoidable in total and by cause. Accomplish- 
ing this task involves summing the avoidable days for each column 
on the Medical Records Analysis Summary and then calculating 
two percentages: (a) avoidable inpatient days divided by total 
sample inpatient days and (b) avoidable inpatient days for each 
cause divided by total avoidable days, according to the formula 
provided on Exhibit 6. . •. . .. 

"With these proportions in hand, the process leader can estimate 
the quantity of avoidable inpatient days incurred by the hospital 
during the sample period in total and by each major cause by 
taking the following steps: . . ■ •. . 

a. Determining from hospital records the total hospital 
inpatient days during the time period covered by the sample. 

b. Determining the number of inpatient days associated with 
the kinds of admissions, discharges, or procedures omitted 
from the sample, e.g. , total hospital days associated with 
normal deliveries. This number often can be obtained from 
hospital data for the period sampled. If actual data are not 
available, the number can be estimated by multiplying the hos- 
pital's total patient days times the ratio of 1) the number of 
patient days associated with cases excluded from the sample 

in choosing the 200 cases to be analyzed, to 2) the sum of 
total patient days in the 200-case sample and the patient days 
associated with cases excluded from the sample in choosing 
the 200 cases. . . - . 

- 22 - 

c. Subtracting "b" from "a. " 

d. Multiplying the remaining days in "c" by the percentage of 
total avoidable days overall and for each case derived from 
sample results. Exhibits 9, 10, and 11 illustrate the range 
of avoidable days of hospitalization in total and by cause for 
three hospitals where CLTR was tested. These results 
provide an estimate of potentially avoidable days for further 
analysis as described in Chapter III. 



The CLTR management group (process leader, physician advisor, 
and administrative liaison) should meet at least twice during the 
course of the records review with other appropriate members of 
the hospital's medical and administrative staffs to discuss the 
results of the review. After 100 cases ha-ve been analyzed the 
process leader should summarize emerging results for prelim- 
inary evaluation and discussion of questions and issues. At a 
second meeting, final results of the records review should be 
presented and agreement reached, if possible, on the extent of 
potentially avoidable hospitalization, the kinds of problems iden- 
tified, the relative priority of those problems, and the major areas 
requiring further study. This agreement sets the stage for the 
second phase of CLTR - the development of realistic plans to 
diminish unnecessary hospitalization and correct any other prob- 
lems identified in the records review. 

- 23 - 


Where the records review described in the previous chapter points 
to relatively little unnecessary hospitalization, this evidence, 
along with other pertinent data (e. g. , low length of stay compared 
with a PAS or other case mix-adjusted standard) could be used by 
the PSRO to exempt a hospital from admissions certification and/or 
continued stay review. In hospitals where there appear to be sig- 
nificant numbers of potentially avoidable days, the CLTR process 
leader, physician advisor, and hospital administrative liaison 
should take the lead in developing and gaining hospital staff commit- 
ment to implement feasible programs to avoid unnecessary hos- 
pital days. Such programs most often will consist of a mix of 
focused review and special actions designed to improve the function- 
ing of the hospital's care activities in areas shown by the records 
review and subsequent analyses to be contributing importantly 
to avoidable inpatient days or other care program problems. 

Developing action programs based on the records review requires: 
(1) designing and carrying out follow-up analyses to identify in 
each apparent problem area the underlying causes of avoidable 
days or other problems; (2) establishing improvement objectives; 
and (3) developing and gaining agreement on cost-effective, im- 
plementable corrective actions. 

IDENTIFY — ^: ^ ■ ■ 


Appropriate and feasible actions to ameliorate major apparent 
causes of unnecessary hospitalization may be clear from the 
results of the records review alone. More likely, follow-up 
studies will be required to identify specific underlying causes 
contributing to each problem area so that targeted action plans to 
deal with them can be designed and evaluated for feasibility. 

For example, it would be important to know whether physician 
management problems (discharge delays, delayed orders on the 
critical path, inappropriate admissions, etc. ) are concentrated 
among certain services or individual practitioners so that review 

- 24 - 

procedures and other corrective actions could be focused on these 
areas. Similarly, if nursing home placement is an apparent prob- 
lem, further study may be needed to determine whether the hospi- 
tal's discharge planning process is less than fully effective or if 
there are simply too few nursing home beds in the area appropriate 
to the physical and financial needs of the hospital's patients await- 
ing placement. Clearly, resolution of such an issue must precede 
development of a program to solve the "nursing home placement" 

The following paragraphs describe possible follow-up studies to 
identify underlying causes of avoidable days attributed in the 
Medical Records Analysis Summary to: (1) physician management 
delays, (2) nursing home placement delays, (3) ancillary service 
delays, (4) failure to provide care on an outpatient rather than 
inpatient basis, and (5) services too infrequently available. These 
approaches were employed or considered in one or more of the 
three hospitals participating in the recent CLTR test. They are, 
of course, intended to be illustrative of possible follow-up analy- 
ses in each area; actual studies should be tailored, as these were, 
to the specific circumstances of each review. 

Physician Management Delays. In order to determine possible 
causes of care delay attributed to physician management in the 
records review,, in one of the demonstration hospitals the process 
leader and records analyst met with several physicians to review 
the diagrams of the cases in the sample which evidenced apparent 
physician management delays. These discussions resulted in 
grouping the apparently avoidable days into three categories: 
(1) days which the physicians agreed were unnecessary; (2) days 
which the physicians felt were necessary but whose necessity was 
not documented in the record (e. g. , family pressure); and (3) days 
which the physician felt were necessary on the basis of a more 
conservative care philosophy than that on which the initial conclu- 
sions were based in the records review. 

Where the physicians agreed that inpatient days were unnecessary 
or where there was inadequate documentation of the reasons for 
continued stay, the interview process itself was felt to have made 
a contribution to correcting the problem. But further action to 
focus review on certain physicians and types of cases identified 
in the interview process was also undertaken. Thus, at this test 

hospital not only did the interviewing help pinpoint underlying 

causes of delay and suggest where action might usefully be taken, 
it also played a part in correcting inappropriate behavior and, by 
familiarizing physicians with the CLTR process, improved their 
acceptance of other corrective action proposals. 

A second approach taken by a process leader at a test hospital 
faced with apparent physician management delays was to conduct 
a profile analysis of the medical staff's active physicians to 
determine whether certain individuals or services experienced 
long average stays, after allowing for case difficulty. For the 
cases managed by these physicians over the preceding 6 months, 
data (length of stay, primary diagnosis, existence of a significant 
secondary diagnosis, whether there was surgery performed, and 
age) were obtained from PAS and other sources to permit com- 
parison of actual length of stay and case mix- specific PAS region- 
al median lengths of stay for each physician. The process leader 
then computed the ratio of actual days to the empirical norms 
represented by the PAS experience, and ranked the physicians by 
this measure. This physician profile analysis was designed to 
provide a basis for a focused review and consultation program to 
ameliorate the physician case management problem. 

Ancillary Service Delays. In another setting in which CLTR was 
tested, a process leader examined more closely an apparent de- 
lay in the provision of routine X-ray services. Discussions with 
the hospital liaison and X-ray department director identified the 
points at which delays might occur (see Exhibit 12). Over a 
week's time (to prevent distortion of results by day of the week), 
times between major steps in the X-ray process were recorded by 
time-stamping each X-ray request slip as it passed a certain point 
in the process. The test was monitored after 2 days to ensure 
that it was running as planned. All of the week's X-ray requests 
were then analyzed and average process times calculated. In 
addition, slips which showed delayed turnaround in relation to the 
CLTR criteria were examined to determine what step contributed 
significantly to the delay. Once these findings were discussed 
with the X-ray department chief and hospital management, specific 
proposals to change several procedures were developed. 

Nursing Home Placement. Nursing home placement delays can 
be a significant cause of apparently unnecessary hospitalization. 
If outplacement delays appear to be a problem, a first step is to 
determine whether unnecessary delays in physicians' initiation 

- 26 - 

and the social service department's processing of nursing home 
placement requests are a contributing factor. To determine 
whether outplacement delays were principally due to internal 
problems, one hospital in the recent demonstration project carried 
out an analysis similar to that described above for identifying 
bottlenecks in the X-ray process. 

After ruling out internal delay as the cause of the nursing home 
placement problem identified in the records sample review, the 
test hospital focused on the reluctance of local nursing homes to 
accept promptly all patients ready for nursing home care. To 
this end, the Social Service Department recorded data (insurance 
coverage, level of care needed, etc. ) every day for 4 weeks on 
each patient awaiting placement in a nursing home. From this 
record, the annual number of extra patient days resulting from 
lack of access to a nursing home bed was estimated by category 
of insurance coverage (or lack of it), by level of care required, 
and in total. Data from this analysis can be used by the PSRO 
and hospital to support an increase in nursing home per diems 
paid by Medicaid in their state. 

In another setting, discrimination by insurance coverage might 
not be the problem so much as a general scarcity of beds in re- 
lation to the size of the over-65 population in the area. One PSRO 
documented the overall shortage of long-term beds by analyzing 
resource and demographic data provided by the local Health Sys- 
tems Agency. To correct the situation, the PSRO and hospital 
could support applications for Certificates of Need for new beds 
filed by local nursing homes. 

Outpatient Substitution. Where a CLTR indicates that a significant 
number of avoidable hospital days is due to inappropriate admis- 
sions, the process leader might tally the admitting diagnoses, 
admitting physicians, and avoidable days associated with each of 
those cases. On the basis of this preliminary evidence, he might 
recommend retrospective review of appropriateness of admission 
for the types of cases or physicians prominent in the tally and, 
where indicated by these studies, pre-admission certification for 
certain types of cases or for the cases of certain physicians. 

If inappropriate pre-admission workup appears from a CLTR to 
be a significant cause of unnecessary hospitalization, a physician 

profile analysis similar to the one described under Physician 
Management, above, could be conducted. However, instead of 
examining each physician's total stay experience against a case- 
matched empirical norm, pre -operative length of stay matched 
by category of surgical procedure would be tallied. In addition, 
if consultation with the physician advisor and others indicates 
certain pre- or post-hospitalization laboratory. X-ray, and other 
services are unavailable on an outpatient basis in the community, 
the process leader might ask the physician advisor to identify 
cases in the ZOO-record sample which could have been shorter if 
outpatient care had been available, list the specific outpatient 
services apparently needed, and estimate latent demand for them. 
With these rough estimates in hand, the process leader might 
develop with hospital management a plan to add outpatient services 
that would be cost-beneficial in terms of unncessary hospitalization 

More Frequently Available Services. Analysis of the 200-case 
sample might suggest that a significant number of hospital days 
could be avoided if certain services were offered more frequently 
(e. g. , nonemergency surgery 6 days per week instead of 5). A 
follow-up study of the cases in which delay appeared for such rea- 
sons could be conducted to determine specifically which services 
were responsible for most of the avoidable days, and on what days 
they were needed. The process leader could then work with the 
hospital administrative liaison and directors of departments sin- 
gled out in the previous analysis to evaluate the additional costs 
(net of hotel cost savings from eliminating some unnecessary hos- 
pitalization) and daily service volume changes likely to result if 
the services were offered on more days. The results of these 
analyses could be used to support the process leader's recommenda- 
tions to the department directors and hospital leadership to expand 
the coverage of certain ancillary services. 

ESTABLISH ' ' ' ■ • . . 


Not all days found to be potentially avoidable from a records sample 
analysis and follow-up studies can be eJiminated through direct 
action by a hospital; some of the causes of avoidable days incurred 
at a hospital will be outside its control. Moreover, some problems 

- 28 - 

may require corrective actions that would cost more (either to 
the hospital or to the patients, where the costs could be passed 
through) than could be saved. In order, then, to set sensible 
and achievable targets for eliminating unnecessary hospitalization, 
the CLTR management group should determine: (1) whether the 
PSRO and hospital (including its medical staff) can act to eliminate 
the potentially avoidable days associated with each cause of delay; 
and (2) whether it would cost more to eliminate the unnecessary 
days than the potentially avoidable days now cost. Three steps 
are necessary: 

1. Categorize avoidable days by susceptibility to action by 
the hospital. Based on further evaluation of the underlying 
causes of avoidable days identified in the records sample 
reviewed, the CLTR management group, working with mem- 
bers of the hospital staff as appropriate, should group the 
avoidable days incurred by the hospital into three categories: 
(a) avoidable days whose causes can be dealt with directly by 
the hospital and its medical staff over the coming year; (b) 
those days that the hospital and medical staff might make a 
contribution to reducing over a longer period of time, but 
which they likely cannot affect in the relatively near term; 
(c) avoidable hospital days whose causes are substantially 
outside the ability of the hospital or its medical staff to 

Most avoidable days due to lack of timeliness in the hospital's 
care programs should fall into the first category. Delays in 
physicians' management of the courses of care for their 
patients might fall into either categories a or b, with differ- 
ences in care philosophy for a given condition being an example 
of a problem that might lead to assigning avoidable days to 
category b. Absence of adequate nursing home beds in the 
community might, under different conditions, cause assign- 
ment of avoidable days to either category b or c. (Of course, 
if the opportunity to establish a nursing home program using 
excess acute care beds at the hospital were available, avoid- 
able days due to this cause might be assigned to category a. ) 
These suggestions are meant to be illustrative; a careful 
evaluation by the CLTR management group will be needed 
to categorize avoidable days in terms of their susceptibility 
to action by the hospital and its staff. 

- 29 - 

2. Frame basic approaches and rough out economics . The 
process of identifying the underlying causes of avoidable 
days and assessing the extent to which the hospital and its 
medical staff could work to reduce them typically will have 
suggested the basic approaches to be taken in each area. In 
this step, these basic approaches should be set down more 
formally, and their implementation costs and savings poten- 
tial compared. 

In making these cost-benefit comparisons, both a system- 
wide and a hospital perspective should be applied. Under 
the cost reimbursement care financing systems prevalent 
throughout the United States, the economic implications of 
avoidable days of hospitalization and of actions to reduce 
those avoidable days may be different for the hospital and for 
the care system as a whole. Careful and cooperative review 
of these economics will be necessary on the part of the hos- 
pital and the PSRO. Moreover, it may be appropriate to 
involve third-party payers in this review. 

The costs that can be eliminated by reductions in avoidable 
days most often will be "hotel costs" of inpatient days when 
treatment was not being rendered or where treatment could 
have been provided on an outpatient basis. The CLTR process 
assumes that treatment accorded patients will continue as 
experienced during the review period. Test experience sug- 
gests that the gross hotel cost saving potential per patient day 
avoided typically will fall into the range of $30-50, but the 
hospital's financial officer should establish a more appropriate 
figure for that particular hospital by dividing its total annual 
patient days into the sum of "hotel function" costs allocated 
to the nursing floors. Only where substantial bed closings 
are programmed are reductions in nursing or other treat- 
ment staff costs likely to be realized. 

3. Set and gain agreement on a target for avoidable days 
reduction. Based on the analyses described in the preceding 
steps, the CLTR management group should pose a challenging 
but realistic target for reduction in avoidable days of hospital- 
ization overall and by major cause for the hospital for the 
coming year. This target should be reviewed extensively 

- 30 - 

with hospital administrative and medical staff, and their 
agreement to it gained. 

With agreement on the target for avoidable days reduction in hand, 
and basic approaches to dealing with the underlying problems 
specified and economically evaluated, the CLTR management 
group will be in a position to move ahead to develop and gain agree- 
ment on a specific corrective action plan. 


For each significant vmderlying cause of avoidable days agreed 
by hospital leadership to be amenable to PSRO and hospital action 
and likely to cost less to correct than it now costs, the CLTR 
management group should "flesh out" the basic approaches deve- 
loped in the previous section into a detailed, step-by-step action 
plan. Actions may deal with problems assigned to either categories 
a or b as determined in Step 1 in the preceding section, although 
avoidable day reduction targets typically should be set only for 
problems in category a. For each proposed action, the plan should 
(a) carefully define the steps to be taken; (b) specify a timetable; 
(c) assign organizational and personal responsibilities for carrying 
out each step; (d) estimate implementation costs, both one-time 
and on-going; and (e) project a measurable impact in the coming 
year in terms of reduced avoidable days and conserved resources 
(e.g. , positions eliminated and dollars saved) by which to monitor 
progress. Exhibit 13 illustrates such a plan. 

Once detailed plans are developed, they should be presented to 
hospital leadership for approval. However, even as the plan is 
being completed, there should be full discussion of the emerging 
actions and their implications to help ensure that a realistic plan 
is prepared and that agreement to it will be forthcoming. The 
plan - objectives and actions - should represent an explicit agree- 
ment between the PSRO and hospital for the coming year, with 
each organization committed to supporting the other in its 

During the implementation period, determining whether the agreed 
upon actions are carried out should not be too difficult; but 

- 31 - 

monitoring results actually achieved will require re -application of 
CLTR to determine whether avoidable days in the areas targeted 
for action have been largely eliminated. If exemption from some 
or all other review responsibilities is agreed upon, substantial, 
demonstrable accomplishment of the plan should be understood as 
a requirement for continued exemption. 

CLTR provides an additional, potentially very useful tool for 
PSROs and hospitals to work together to reduce avoidable days of 
hospitalization. Realizing its potential will require imaginative 
and collaborative application of the procedures described in this 
document by administrative and medical leadership in PSROs and 
hospitals across the country. 

- 32 - 



s riaiHXj 








A. Analyzing care 


Based on Test Experience 

V'.l Required Staff Hours '' Vf'*' -""f'- 

1. Designate key participants >q 




2. Set review criteria ^nljuoR 

3. Categorize possible causes of jgtg 
avoidable stay .baq?. 


4. Specify the sample structure ; aar, 

5. Select the sample cases ,,;,o; lariA 

6. Conduct records review 

~ ; ■ ■I-;-, ciTin? 

7. Summarize results and extropolate 
to hospital 

8. Present findings to hospital ■ - ' 

Phase I Sub-total 


iXfln 2 





1 j 



B. Developing corrective action plans 

1. Identify underlying causes 

2. Establish improvement objectives 

3. Develop and agree on corrective 

Phase II Sub- total 

CLTR Total 















Weekly Timetable 












9!5r!w Y'>n9Qi9'Tf>nol'l 

litnu bsjfjibfii ton Yisie''u2 

or fiotsiosb i/div/ijli'o- vKb 

A '6!iLii-oH as srnbS 

A iK'.ioKiH 26 3mt>2 

-I'd boifjh-'O v&Cl 


33 - 

- i^i - 


(Unless noted, weekdays are Monday-Saturday) 





1. Physician will order test, 
procedure, consult, or 

First weekday clinically in- 
dicated by test result, con- 
sult report, procedure 
completion, or observed 
condition of patient 

Same as Hospital A 

Same as Hospital A 

2. Laboratory test results 
will be available for 
physician review 

Routine: same day if 
ordered in a.m. or next 
weekday if ordered in p.m. 

Special (cultures, Australian 
antigen): Iweekoras 

Routine: same day if 
ordered in a.m. or next 
weekday if ordered in p.m. 

Cultures, cardiac enzymes: 
3 days or as indicated 

Viral and hormone studies: 
1 week 

T3/r4, HAA: 5 days 

Routine: next weekday 
Stat or Today Please: end 
of day 

Pre-operative: evening 
rounds same day 

Cultures: preliminary re- 
port by second morning, 
significant findings as 
they occur 

Special tests: several days 
as per lab schedule 

3. Diagnostic x-ray will be 
available for physician 

Routine: same day or next 

Special: second day or 
next weekday 

Routine: same day or next 

Special: same day or next 

Routine: same day or next 

Stat or Pre-op: same day 
Special procedure (IVP, Gl, 

contrast): next weekday; 

one per weekday if 

multiple procedure 
Arteriogram: next day or 

next weekday 

4. Electrocardiogram re- 
sults will be available 
for physician review 

Next weekday 

Next weekday 

Same day if ordered in a.m., 
otherwise next day 

5. Electroencephalogram 
results will be available 
for physician review 

Next weekday 

Next weekday 

Next day 

6. Consults and referrals 
will be performed and 

Same day or next weekday 

Same day or next weekday 

Within 24 hours of 

7. Operative procedures 
will be performed 

First day OR available to 
service after decision to 
operate unless consent form 
delays, medical complica- 
tion arises, etc. 

Next weekday following 
decision to operate or as 
scheduled pre-admission 
unless consent form de- 
lays, medical complication 
arises, etc. 

Elective cases (except colon): 
day following admission 

Elective colon cases: 
third day following 

Emergency cases: same day 
as indicated 

Non-emergency where 
surgery not indicated until 
hospitalized: second week- 
day following decision to 

8. Discharge will be 
carried out 

Day ordered by 

Same as Hospital A 

Same as Hospital A 

- 34 - 



1. Part or all of treatment could have been as outpatient 

2. Physician management 

a. Delay in ordering something in critical path 

b. Delay in ordering discharge 

'9<f0 i 3. Consult delay 

\ -i . — 4. Inadequate pre-admission scheduling 

L»^^ ; a. Of diagnostic workup 

b. Of OR time 


1. Diagnostic radiology delay 

a. Routine 

b. Special 

2. Nuclear medicine delay "\ 

3. Clinical laboratory/pathology delay 

a. Routine 

b. Special 

4. Surgery delay 

a. OR preferred time 

b. OR capacity constraints 

5. Nursing home placement delay 

a. Social service 

b. Lack of nursing home bed of funding 

6. Administrative discharge delay 


1. Patient/family pressure 

2. Teaching 

3. Research 

4. Other (specify) 


- 36 - 




Sample Number: 46 
Admission/Discharge Number: 51322 
Chief Complaints: Tenderness, abdomen 
Admitting Diagnoses: Possible urinary tract infection 

Final Diagnoses: 

1. Urinary tract infection 

2. Otitis media 

3. Hemorrhoid 

Age: 31 

Residence: Cityville, U.S.A. 
Date Admitted: 6/29/77 
Service Admitted: Medicine 
Admitting Physician: Dr. T. 
Attending Physician: Dr. T. 
Surgeon: Dr. T. 
Date Discharged: Home 
Length of Stay: 1 6d 
Insurance Coverage: Medicaid 

Operative Procedures: 

Cystogram, cystourethroscopy 
urethral dilatation 







- 37 - 


















- 38 - 



Sample Number: 55 
Admission/Discharge Number: 10480 
Chief Complaints: See Final Dx 
Admitting Diagnoses: See Final Dx 

Final Diagnoses: 1. Cerebrovascular accident with leftside 

2. Arteriosclerotic cerebrovascular 
disease with cerebral ischemia 

3. Diabetes mellitus 

Operative Procedures: None 

Age: 71 

Residence: Townville, U. S. A. ' ' ■ 

Date Admitted: 1/28/76 

Service Admitted: Medicine 

Admitting Physician: Dr. T 

Attending Physician: Dr. T 

Surgeon: None 

Date Discharged: 2/17/76 

Discharged To: Pinewood Chronic Hospital 

Length of Stay: 20d 

Insurance Coverage: Medicare 











Skull X-R ORD/RTD 


Physical Therapy 
Social Service - ORD 

Brain Scan ORD/RTD 

Consult Dr. A. 

FBS 7.d. 



PT Consult RTD & Therapy begun 

Social Service RTD - Form 256R initiated -■ 
FBS, LYTES, ALK Phosph, Cholest Tri Gly, 
Albumin, Globulin ORD/RTD 

Consult Dr. A. RTD -« 







MD. Notes: Patient to be up 

Form 256R ►Health Dept. 

SS note: awaiting bed 



Notification ■ 

bed available 2/17 

Trans to Pinewood 
"Continue PT" 

Notes: 1. No real change in FBS regardless of 
insulin dosage 

2. Patient made minimal progress in 

3. Have Physician Advisor determine 
maximum hospital benefit 

Id delay-physical 
therapy response 
Id delay- SS 
answer request 

Physician advisor 
feels maximum 
hospital benefit 
on approx. 2/1 

13d awaiting 


Od impact 

Id placement 

13d placement 
delay — Lack 
of bed 


- 39 - 



Sample Number: 22 
Admission/Discharge Number: 22368 
Chief Complaints: — 

Admitting Diagnoses: Diabetes mellitus, ASCVD, 
Organic brain syndrome, Osteoarthritis 

Final Diagnoses: Diabetes mellitus 

Peripheral neuropathy and neuritis 2° to Dx #1 
Cerebral arteriosclerosis w/Organic brain syndrome 
Mixed arthritis 

Operative Procedures: None 

Age: 70 

Residence: Big City, U.S.A. 
Date Admitted: 5/21/76 
Service Admitted: Medicine 
Admitting Physician: Dr. F. 
Attending Physician: Dr. F. 
Surgeon: None 
Date Discharged: 6/12/76 
Discharged To: Home 
Length of Stay: 22d 
Insurance Coverage: Medicare 




















































FBS daily rtd 
CXR - ord/rtd 
EKG - ord/rtd 
Electrolytes ord ■ 


Pt hallucinating 

Ca, P, Alk Phosphatase, SCOT, SGPT, 
CPK, Cholesterol, A/g] ord — j 

Bid series rtd 
Electrolytes rtd 

Flat plate ABD ord 

Flat plate ABD rtd- 
GI and GB series ord/rtd 

3 —I 

Flat plate ABD rtd- 
GI and GB series or 

Pt confused, disoriented, BS still t , intake poor 

L-S spine XR ord - 
Consult Dr. C. ord- 

Dietary instructions 
L-S spine XR rtd -• — 

Consult Dr. C. rtd- 
Soft L-S binder 

Cooperative, alert 
C /O leg pain 

BS stabilizing 

2d delay 

3d delay 
ord PT 

PT and exercises - ord/rtd 
\ Tolinase 
\ Tolinase 


2d delay D/C 

Physician advisor 
feels Day 21 D/C 


No impact 
because BS 
stiU out of 

No impact 
because BS 
still out of 

2d physician 




- 40 - 


Hospital A 

(January-December 1976) 
172,305 days = 100% 

141,300 days = 100% 

17,000 days = 100% 

room delay 

- 41 - 


Hospital B 

(Dec. 1975-Nov. 1976) 
19,442 days = 100% 

16,800 days = 100% 

2,500 days = 100% 

100 -Other 


Patient could have 
been treated in 
whole as outpatient 

- 42 - 



Hospital C 



62,784 days = 100% 55,200 days = 100% 5400 days = 100% 


Physician delay in 
ordering discharge 

- 43 - 





Doctor Order Patient Patient 

writes 1 hour received 1 hour called to 5 hours leaves floor 
order at X-ray X-ray for X-ray 

.5 hour 

Evaluation Film Procedure Patient j 

dictated 10 hours available 1 hour completed 2 hours arrives at -* — « 
for viewing X-ray 

24 Hours 

Evaluation Evaluation Evaluation Evaluation 

transcribed 1 hour leaves X-ray 3 hour s arrives on 4 hours ^ on chart 
forward onward 

- 44 - 



(Illustrative - Based on Actions 
Developed in Test Hospitals) 

Major Problem Area Underlying Cause 

1. Patients could have 
been treated as 

2. Physician case 
management delays 

Outpatient clinic 
staff capacity 
inadequate on 
evenings and 

Ten physicians • 
kept patients more 
than 50 percent 
longer than PAS 
median on average 
and accounted 
ately for physician 
management de- 
lays in 200-case 

Ptenned Steps 

a. Add 1.0 fte RN and 
0.5 clerical position 

b. Assign one extra resi- 
dent to cover outpa- 
tient clinic on evening 
and weekends (27 
hours per week) 

a. Interview 10 physi- 
cians to explain 
findings and need 
for more stringent 

. Double frequency of 
concurrent review for 
these 10 physicians 
for 6 months, then 
repeat profile analysis 
on these 1 physi- 

Deadline Responsibility 

Potential Resources Saved (Required) 
One Time Annually Comments 

7/1/77 Director, OP Dept. 

7/1/77 Chief of Medicine 

7/1 5/77 Physician Advisor 
and Chairman of 
Utilization Review 


Records review 


( 6,000) 

Saves 1,000 patient 
days (frees 3 beds) 

Saves 1 ,500 patient 
days if half of "LOS 
gap" for 10 physicians 
closed in first year 
(frees 4 beds) 

( 5,000) 

3. Nursing home place- 
ment delay 

Superfluous 30-bed 
nursing unit in old 
wing and need for 
space for non- 
patient functions, 
e.g. nursing ad- 

To few Med i- 
caid beds in 

a. Support Certificate 
of Need Application 
of Smith Nursing 
Home if 7 beds set 
aside for this hospi- 
tal's Medicaid 

b. Support increase in 
Medicaid per diem 

8/12/77 + 

Eliminating 2,500 
patient days by 
corrective actions 
listed above cuts 
average daily cen- 
sus by 7 patients; 
23 other beds can 
be closed without 
raising medical 
and surgical oc- 
cupancy above 

9/1 5/77 ^ 


, Close 30-bed unit 
transferring patients 
and staff to other 
units and freeing 
space for non-patient 

, Eliminate 7 posi- 
tions and $60,000 in 
annual payroll and 
benefit expense 
through attrition, 
as follows: 

2 ward clerks 

2 housekeepers 

1 tray carrier 

1 supply deliverer 

1 kitchen aide 

Eliminate $15,000 in 10/30/77 
annual food and medi- 
cal supplies cost 

1. Re-allocate $15,000 
in annual utility, re- 
pair, and building 
depreciation expense 
from nursing unit 
to non-patient 
cost centers 

Hospital Director 
PSRO Executive 

Hospital Director 
PSRO Executive 

Director, Nursing 

If Smith carries out, 
could save 2,500 
patient days (freeing 
7 hospital beds) 

If 7 more beds are 
made available to this 
hospital's Medicaid 
patients, 2,500 pa- 
tient days could be 
saved (freeing 7 
hospital beds) 

Made possible by 
actions to eliminate 
avoidable days de- 
scribed In 1 and 2 

10/30/77 Hospital Director 

Manager of 

10/30/77 Controller 



Potential Net Saving 








This appendix supplements the brief discussion of sample size 
and case selection presented in Chapter II, Sections 4 and 5, of 
the main body of this document. 


The number of cases needed to be analyzed in carrying out CLTR 
is dependent on two countervailing considerations. The more 
cases which are analyzed, the more confident one can be in the 
statistical validity of the findings; but the more time-consuming 
wEl be the CLTR process. The CLTR demonstration project has 
indicated that a 2 00 -case sample offers a reasonable balance 
between statistical validity and resource economy. 

There is a direct relationship between sample size and the statis- 
tical validity of findings based on analysis of the sample. Statis- 
ticians have determined that one can be 90 percent confident that 
the percentage of avoidable days (p) estimated from an n-case 
sample is within e percentage points of the true avoidable days 
percentage for all cases from which the sample was drawn, where 
e = 1. 645 \Jp ( 100 - p)/n. * The two examples which follow illus- 
trate how this relationship can be useful to the user of CLTR. 

Example 1: With a sample of 200 records - the number used 
in previous demonstrations of the CLTR methodology - we 
can be confident that an estimate of 10 percent of days being 
avoidable is within 3. 5 percentage points of the true percent- 
age of avoidable days for all of that hospital's cases from 
which the sample was drav.Ti, since 

e = 1. 645 n/10 (100 - 10)/200 = 3. 5. Exhibit A - 1 portrays 
graphically the relationship between sample size and 90 
percent confidence limits for various estimates of the percent- 
age of avoidable days. 

* - Derived from the sample size estimation fiinction for 
binomial parameters. 

- 46 - 

Example 2 ; If analysis of the 200 -case sample indicates 
that only 2 percent of total sample days are avoidable be- 
cause of problems with nursing home placement, we can 
be 90 percent confident that the true percentage of days 
avoidable for that reason among the cases from which the 
sample was drawn lies between 0. 4 percent and 3. 6 per- 
cent. Since e = 1. 645 \/2 (100-2)/200 = 1.6 and 2 ±1.6 is 
0. 4 or 3. 6. Exhibit A - 2 graphs the spread of likely true 
avoidable day percentages for various estimates based on a 
review of 200 cases. 

As long as the number of cases in the sample is less than 5 per- 
cent of the cases from which the sample is drawn, the relation- 
ship between likely estimation error and sample size described 
above is unaffected by the number of cases from which the sample 
is drawn. Thus, a 200 -case sample is just as valid a representa- 
tion of what has transpired over a year in a hospital which had 
4, 000 discharges (about 75 beds) as in one which had 40, 000 cases 
(about 750 beds). Even smaller hospitals than those with 4, 000 
annual discharges can be analyzed by examining 200 cases if the 
sample is drawn from more than 1 year's discharges. 

Since the principal value of CLTR is in determining which hospi- 
tals have major opportunities to avoid unnecessary hospitalization 
and to identify the two or three most significant causes of avoidable 
days, it is not necessary to achieve single-percentage-point preci- 
sion in estimating avoidable -days percentages. It would certainly 
not be economical to do so. For the range of total avoidable-day 
percentages likely to be found in most community hospitals - perhaps 
5 to 15 percent - a sample size of 200 is sensible. With 200 cases, 
a 10 percent estimate is within 3. 5 points of the true value (Exhibit 
A - 2). To cut the margin of error in estimating an avoidable-days 
percentage of 10 percent from 3. 5 percentage points for 200 cases 
to 2 percentage points would require analysis of an additional 135 
cases, and a 1, 730-case sample would be required to ensure that a 
10 percent estimate was within 1 percentage point of the true value. 
The recent test suggests that the additional time and cost involved 
in reviewing these larger samples would not be justified by enhanced 
accuracy or credibility in the results. 

- 47 - 


Selecting the initial list of the 300 admissions or discharges 
needed to ensure a representative sample, and narrowing that 
list down to a final sample of about 200 records, is a critical 
step in the CLTR process. The records analyst should work 
from a comprehensive list of admissions or discharges for the 
designated sample period. Where the analyst has a choice of 
either an admissions or discharges list, the discharges list is 
preferred to ensure that all cases listed have been completed at 
the time of the selection. The list used must indicate the medical 
record number for each case, or some other information, e. g. , 
patient name, from which the medical record describing the case 
episode can be identified. 

The cases in the comprehensive list will often be numbered con- 
secutively; if they are not, the records analyst should number 
them consecutively beginning with 1 - or at least be able to 
determine which number each case would have if they were so 
numbered. For example, if the discharge list is initially not 
mombered consecutively, the records analyst should count the 
cases in the list, writing the "count" at every tenth case next to 
it on the listing. K the cases are initially numbered consecutively, 
but simply do not begin with 1, the analyst need not count the cases 
at all. Rather, he can assign to each case the new n\imber equal 
to its old number plus 1 minus the lowest old number in the list. 
Thus, a case list numbered consecutively from 49, 606 to 57, 403 
would be assigned new numbers from 1 to 57,403 + 1 - 49, 606 = 
7, 798. 

Once the comprehensive list of cases for the period to be sampled 
is numbered consecutively, the records analyst should acquire 
a sample of 300 records from the list in either of two ways: (1) 
by generating one on a computer for the range of numbers spanned 
by the comprehensive list of admissions or discharges or (2) by 
generating one manually from a printed random number table. 
If the hospital or PSRO has access to a time-sharing computer 
system, the computer-run table is easily generated through con- 
sioltation with the user's manual or service representative. In 
the case of a discharge list which initially is numbered consecu- 
tively but does not begin with one, it is often possible to generate 
by computer a random number table for the initial span of numbers 

- 48 - 

(49, 606 to 57, 403 in the example above) without renumbering the 
list beginning with 1. 

If computer generation of a suitable random number table is not 
feasible, the records analyst should refer to the random number 
table in Exhibit A - 3 or some other found in a statistics textbook. 
First, the analyst should co\int the digits in the number of the last 
discharge in the list. If there were 24, 000 discharges in the list, 
they would be numbered 1 to 24, 000, and the number of digits in 
the last nvimber (24, 000) would be 5, Next, the analyst should 
draw vertical lines separating the random number table into 
columns of as many digits as there were digits in the number of 
the last discharge listed. Once this is done, the analyst should 
scan each column of the table, drawing boxes around the first 300 
numbers in the table which are encompassed by the numbers of 
cases in the discharge list. For example, for the 24, 000-case 
example described above, the first three cases taken from the 
random number table in Exhibit A - 3 would be 10,480, 22, 368 and 
9, 429 - the first numbers in the random table falling vidthin the 
range of 1-24, 000. 

The initial sample of 300 cases should be reduced to 200 following 
the procedures described in the body of this document. 

- 49 - 



+10% r 

Interval (e)* 
in percentage 

With 200-case 
sample and 
avoidable days 
percentage of 
10%, true 
percentage has 
90% likelihood 
of lying in the 
range 10±3.5% 

Estimates of 
avoidable days 


Sample size (n) 
in cases 

* 690 = 7. 645 \/p(100- pj/n , Mhere p is the estimated percentage of avoidable days, n is the number of cases 

in the sample, and Ptfigg « f'le range in which there is a 90% probability that the true percentage of avoidable days lies. 

- 50 - 




Range of 90% 
Likelihood for 
True Avoidable 
Days Percentage 

If sample 
voidable days 
lercentage is 
'10%, true 
percentage has 
90% likelihood 
of falling in the 
range 1Q±3.5% 


Estimated Avoidable 
Days Percentage (p) 

* = 1.645 \/p{100—p)/n , where p is the estimated percentage of avoidable aays, n is the number of cases 

in the sample, and P+egg is the range in which there is a 90% probability that the true percentage of avoidable days lies. 


- 51 



10 43 15011015360201 18164791646691791419462S9036207S0969 9 9 5709129190700 
g a 3 6 e| 4 6573855958539330995B9198279e253403939653409552666191743961599505 
241304836022527973657639 36 46091S179!5483049 3 4032OB13O680196556334B58fi29 
42167930930624361680078561637639440535377134157004 0t)849749179775fll6379 

99562729055642069994988723101671194187384401348640638132) 691063412952 
89579143426366110261174531810357740843782!) 3311256658678449470 5 58556941 

63553 409614 8235034274962 66 9445186637269552160208471223490511337039 322 
109 4 2 91 9 39 69526369273788974334883632017617300150827284115371563061374952 
10365611298752985689482375226767689933940151126358851042028 5 29975fl9H68 
07119973367104808178772331391647564810569773565977293 7 2744612855190707 

02368213825240460268893681988555332440190118865255648354491905044555 57 
010115409233362949 431273041461859429652715858503051133019159274764951 
5216253916463695H?«62321614513831499873633495643509473817752351 5 635749 

4866391245858261434609172301689022 90 4734591933217830421616669990432H13 
024863306228834073511973192420609 5 261280500016765833586866795072094953 

0«74 25 7 3933906466432B46734002732832613629H9479606764 7 C 0645849609698253 
9193126418641179430526766 2 5 9403997222209715006456891402424160784469618 

0058204711879177734 142206351267406799547818174260743808766556202876630 
0072569884687975 61706632468 7 27 622236086646379316176038656557791968006 
690116579595876533931698827354365750862540801599202 9841801501277748501 

46503185841884549618023045103 8 2 065558727381681547556943533892056287338 

9648707523333626 457001636924776696998420046804558546565041024688045709 
3491463976867 2 06806534476170338758940836324277000270663888637777569348 
5 3 97654914069906 7 24568350829481139642878802878826747363466340654197809 
76072 2.9 5154698007191587452 5 77422987600593991196169411511422260697595B3 

089620035631663253886164234072812493564656891693524837 34 55767654781788 
95012683799352670765105920454 27 646354328023491784728865147776273092277 
1566410493204933839191138219995951 68 1652371954 8 aa34675ia39a3322«185653 

- 52 -